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Abstract 

This  paper  gives  tight  upper  and  lower  bounds  for  the  minimum  area 
required  to  sort  n  it-bit  numbers  in  a  digital  medium,  when  the  inputs  can  be 
replicated  up  to  r^n  times.  We  also  give  provably  good  AT~  bounds  for  VLSI 
sorting  circuits  that  read  their  inputs  once.   Among  other  results,  we  prove: 

(1)    The  minimum  area  A{n,k,r)  required  to  sort  n  ik-bit  numbers,  which  can  be 
read  r  times,  satisfies 


A{n,k,r)  =  0 


'logn  +  2*log-5-;-,       2*<n/r, 

logn  +  -^log-^^^.        n/r<2*<n^(^\ 


(2)    A  VLSI  circuit  that  sorts  n  Jk-bit  numbers,  for  n<2*<n*^(^\  and  that  reads  its 
inputs  exactly  once  satisfies 

AT~=n{n^\og^- ). 

n 

1.   Introduction 

"Optimal  Area  VLSI  Circuits  for  Sorting"  [S]  analyzes  the  problem  of  how 
to  sort  n  ^-bit  numbers  in  a  minimum  area  VLSI  circuit.  Lower  bounds  for  the 
circuit  area  are  established,  and  sorting  circuits  are  described,  which  require  an 
area  that  is  within  a  constant  factor  of  optimal.  These  results  are  based  on  the 
fact  that  the  inputs  are  read  only  once.  We  now  analyze  the  problem  when  the 
inputs  are  read  r</i  times.  This  problem  is  clearly  more  difficult,  and  its  solution 
requires  new  techniques  to  measure  how  much  information  a  sorting  circuit  must 
be  able  to  retain. 

In  [S],  bounds  are  established  by  showing  that  a  circuit,  at  times,  has  to  store 
a  certain  amount  of  information,  lest  it  fail  to  sort  correctly.  Also,  a  Testing 
Lemma  is  used  to  verify  the  presence  of  stored  information.  It  is  based  on  "test- 
ing", the  ability  to  assign  a  variety  of  different  values  to  bits  which  have  not  yet 
been  input  to  the  circuit  by  a  specific  time.  The  advantage  in  testing  is  that  it 
provides  a  simple  alternative  to  the  sometimes  arduous  task  of  constructing  fool- 
ing sets,  as  formalized  in  [U].  Unfortunately,  these  simple  techniques  seem  to  be 
inapplicable  when  inputs  are  read  more  than  once.  In  this  paper,  proofs  are  there- 
fore based  exclusively  on  fooling  set  constructions. 
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This  choice  of  proof  style  has  advantages.  The  techniques  are  more  general, 
and,  we  believe,  more  insightful.  Theorems  2  and  3,  for  example,  each  indepen- 
dently prove  that  any  VLSI  circuit  which  sorts  n  ifc-bit  numbers,  for  n^2*</i^(^\ 

2^+1 

satisfies  AT~^Cl{n-\og- ).    The  circuit  is  required  to  be  semellective,  {i.e,  it 

n 

can  read  the  input  data  only  once,)  and  to  have  a  when-  and  where-determinate 
i/o  schedule.  We  also  explain  why  this  result  is  the  best  possible  for  the  stzmdard 
one-cut,  total  information  flow  proof  technique. 

In  our  circuit  model,  at  most  one  bit  can  be  stored  per  square  unit  of  area, 
and  i/o  ports  also  require  a  unit  of  area.  The  i/o  schedule  is  assumed  to  be  when- 
and  where- determinate.  Determinate  schedules,  which  require  prespecified  times 
and  locations  for  the  input  and  output  of  each  bit,  are  discussed  in  [U],  and  for- 
malized in  [Ke].  Our  area  bounds  are  proved  for  multilective  input;  that  is,  data 
can  be  read  more  than  once.  All  computations  and  temporary  storage  of  data, 
however,  must  be  done  within  the  sorting  device. 

The  input  variables  are  named  X^,  Xo,  ■  ■  ■  ,X„.  The  sorted  output  variables 
are  Yi,  .  .  .  ,Y„,  where  Yi^Yj^  ■  ■  ■  ^Y„.  The  bits  of  Yj  are  Yjj,  where  Yj^  is 
the  most  significant  bit  (msb).  The  msb  is  sometimes  called  the  I'st  highest  order 
bit,  and  y^ ^  is  called  a  I'st  least  significant  bit  (Isb). 

2.   Lower  Bounds  for  Area 

To  simplify  the  exjxKition,  we  shall  assume,  in  this  Section,  that  the  i/o 
schedule  inputs  and  outputs  at  most  one  bit  per  time  step.  Since  i/o  pads  consume 
unit  area,  concurrent  inputs  can  be  viewed  as  serial  inputs  into  a  shift  register. 
Our  area  (only)  bounds  never  include  area  consumed  by  wires,  and  consequently 
there  is  no  cost  attributed  to  routing  the  stored  inputs  to  their  destinations.  Thus 
the  area  bounds  for  the  simplified  schedules  also  apply  to  circuits  using  arbitrary 
determinate  schedules.  Similarly,  we  may  assimie  that  an  input  and  an  output 
never  occur  at  the  same  time  step.  It  should  be  noted  that  the  input  schedule  is 
not  assumed  to  be,  say,  r  copies  of  a  schedule  where  each  input  occurs  once. 
Each  bit  of  an  input  variable  can  have  its  r  input  instances  read  in  any  deter- 
minate way,  and  could,  of  course,  have  fewer  than  r  instances. 

Theorem  1. 

Suppose  each  input  is  read  r  times,  and  2*s  — .  Then  A  =  n(2*log — 73 r)- 

Proof:  It  suffices  to  assume  that  I'^s^n/lr.  Choose  t={'t-[,  tJ  so  that  for 
Ti</<T2,  at  most  nfl  least  significant  bits  are  input,  and  n/2r  Isb's  are  output 
during  ti</^T2.  Without  loss  of  generality,  we  may  assume  that  n/Ar  of  those 
output  bits  are  variables  y,j^,  where  i^n/2.  (An  analogous  argimient  holds  for 
i>n/2.)  Let  those  n/4r  Isb's  comprise  the  set  of  variables 
^  =  {Yi^ic,Yi^ic,  .  .  .  .Yi^^k).  Name  the  inputs  so  that  the  least  significant  bits  read 
during  t  are  X^^,  X2fc,  .  .  .  .X^^k^  where  a^n/2.  By  assigning  X,  =  2*-l  for 
l^/'^a,  we  can  guarantee  that  the  outputs  in  ^  are  independent  of  the  Isb's  read 
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during  t.  The  circuit  must  therefore  have  a  memory  at  least  as  large  as  the  infor- 
mation content  of  the  outputs  of  these  n/4r  least  significant  K-bits.  Formally,  we 
assign  values  to  the  remaining  {Xj}j=c^+i  in  batches.  Put  /o=0,  and  let  //-//_i  of 
these  X's  equal  ^/,  for  /=l,2,...,n/4r.  Set  the  remaining  n- a- 1^/4  X's  to  2*-l. 
Thus  1',,  =  ^/.  Now  the  chip  need  not  "remember"  n/4r  distinct  assignments  to  the 
Isb  input  variables  ^,^,  because  many  of  the  ^  variables  must  have  identical  values 
(if  2'^«n/2r).  We  assign  values  to  the  4's  so  that  4;  has  the  number 
j  (mod  2^^"^)  comprising  its  Jt-1  most  significant  bits,  for  j  =  l,2,...,n/4r.    By 

construction,  there  are,  for  x=0,l,...,2*~^  — 1,  about r—r  inputs  which  have  x 

4r2'^    ' 
assigned  to  the  k-1  most  significant  bits  (and  which  have  as  yet  unspecified 
Isb's).    Thus  for  each  x,  the  number  of  output  variables  in  ^,  which  have  x  so 
assigned,  and  which  have  0  assigned  to  the  least  significant  bit,  can  be  any  value 


from  0  to  — -j^.   Therefore  the  number  of  such  distinct  outputs  is  at  least 

1+     " 


— .    inereiore  tne  numoer  oi 

2*-! 

2^+1" 

whence  taking  the  logarithm  gives,  fl(2*log — rrr).  n 

r2     '■ 

Theorem  2. 
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Suppose    each    input    is    read    r    times,     and     — <2*<(— )^.     Then 

r  n 

Proof:  Write  r]  =  n/r,  K  =  logTi,  and  k^K+f.  In  terms  of  the  new  variables,  the 
area  bound  can  be  written  as  A  =  [l(-r\f).  It  thus  suffices  to  have/<€K,  where  e  is 
positive  and  will  be  specified  later.  In  view  of  Theorem  1,  we  may  also  assume 
that  f>6. 

We  want  to  find  a  time  interval  when  many  bits  belonging  to  the  /  lowest 
order  positions  are  output,  but  no  more  than  half  of  the  higher  order  bits  are 
read.  An  area  bound  can  then  be  found  by  computing  how  much  additional 
information  must  be  remembered  by  the  circuit,  if  its  outputs  are  to  be  correct. 
Since  there  will  be  no  restrictions  on  which  lower  order  bits  are  read  during  the 
interval,  we  bound  the  uncertainty  in  the  output  by  the  information  content^ 
/(the  lower  order  outputs  |  all  X-inputs  except  for  —  higher  order  X-bits). 

The  difficulties  in  estimating  this  expression  are  that  the  given  (conditional) 
information  can  be  much  larger  than  the  information  content  of  the  output  bits, 
that  the  output  bits  can  have  a  skewed  density  among  the  Y  variables,  and  that  a 
large  value  of  r  can  force  it<2K  to  be  so  small  that  the  X's  can  take  on  relatively 

'This  expression  is  actually  a  function.  Since  its  principal  purpose  is  motivational,  we  omit  a  formal  definition;  the 
details  are  not  essential  to  our  proof.  More  about  /  can  be  deduced  from  [U]  and  the  remainder  of  this  Section. 
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few  values.  Thus  most  lower  order  output  bits  can  be  highly  correlated  with  their 
neighbors. 

The  proof  has  five  steps: 

(1)  Find  Ti^  lower  order  output  bits  which  can  have  high  information  content 
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(despite  their  sorting  constraint)  and  which  are  output  during  a  time  when  at 
most  -r-  higher  order  bits  are  input. 

(2)  Use  step  1  to  assign  values  to  (various)  higher  order  bits  of  all  n  F's. 

(3)  Choose  a  large  subset  of  the  input  variables  which  have  many  higher  order 
bit  positions  that  are  not  read  during  the  time  interval  of  step  1 . 

(4)  Assign  values  to  the  input  bits  so  that  the  unread  bit  values  of  the  X's  impli- 
citly partition  many  of  the  output  variables  into  equivalence  classes. 

(5)  Show  that  because  of  the  higher  order  X-bits  which  are  not  input  and  the 
lower  order  bit  assignments  to  the  X's,  the  uncertainty  of  the  lower  order  bit 
values  for  the  y's  within  each  equivalence  class  is  high. 

Let  T;,,  /i  =  0,l,--,6r  be  a  partition  of  time  so  that  during  each  time  interval 

f€(T;„  T;,+i],    "^   bits  are  output  which  are  of  k  +  1   to  k  significance.     Let 
6 

C;,=         2         2~^.   C;,  is,  essentially,  the  cost  of  having  those  lesser  significant 

yj,  output  during 

bits  be  independent. 

Step  1.  Clearly    2  ^a  =     2       2   2~'<—  =  r,  so  at  least  Ir  of  the  C;,'s 

have  values  less  than  1/4.   Furthermore,  during  some  time  interval  t  =  (t;,,  t;,+  i] 

corresponding  to  one  of  those  Q's,  no  more  than  -r-  bits  can  be  read,  which  are 

of  1  to  K  significance,  since  the  more  significant  bits  are  read  a  total  of  /tjk  times. 
Let  that  value  of  C;,  be  C.  Let  Sj  be  the  set  of  lower  order  bits  of  Y^  which  are 
output  during  t,  for  _/  =  l,...,n.   The  5/s  are  the  lower  order  bits  of  step  \. 

Step  2.  We  now  assign  values  to  the  higher  order  bit  positions  of  the  outputs 
Y.  Let  Sj  =  ifc+l-min{oo,  h^S^i.  We  temporarily  set  71  =  0,  and  momentarily 
put,  for 7  =  2,3,..., n 


^r^ 


Yj.^^l'>-\  if  Sj.^^s 


Yj.^^l'>-  (y,_i  (mod  2'0).      if  Sj- 


'], 


1  <  Sj. 


Evidently  y„<2^'''-C  <  2*,  so  the  K's  take  on  permissible  values.  Note  also  that 
Yj  =  Yj-1  only  when  Sj=Sj-\  =  <^.  Furthermore,  the  Sj  least  significant  bits  of  Yj 
can  be  altered  arbitrarily  without  affecting  the  ordering  of  the  K's.  Let  the  set  Y 
consist  of  all  the  Y/s  with  Sj>0.    Any  YjeY  currently  has  its  Sj  lower  order  bits 


;  -mi.     "C 
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set  to  0.  These  lower  order  assignments  will  be  altered  in  Step  4.  The  higher 
order  assignments  are  permanent. 

Step  3  begins  with  the  observation  that  there  are  at  least  —  input  variables, 
each  having  at  least  —  higher  order  bits  which  are  not  input  during  t.  Now  the 
number  of  different  ways  —  bits  can  be  distributed  among  the  k  higher  order 

positions  is  only  (^3)<('n)'°2^"^'-'<(Ti)-^^.  K  the  X's  are  partitioned  into 
"pools",  where  pool  members  have  an  identical  set  of  —  unread  higher  order  bit 

positions,  then  the  mean  pool  size  is  at  least  —n-^r^^.    If  we  consider  only  X's 

belonging  to  pools  of  size  —n-^^r-^^  or  more,  then  at  least  —  X's  remain. 

8  8 

Step  4.    Values  can  now  be  assigned  to  the  X's.  It  is  convenient  to  suppose 

that  there  are  in  fact  more  than  n  "good"  ones,  rather  tham  just  — .    Thus  we 
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suppress  some  unimportant  details  by  imagining  that  each  pool  is  replicated  a 
small  number  of  times,  say,  up  to  16  copies.  Later  amends  will  trivially  rectify 
this  overcoimt. 

Our  strategy  is  to  assign  values  to  almost  all  the  X's  in  a  pool  before  going 
on  to  use  the  members  in  another  pool.  Accordingly,  we  pick  a  pool  of  X's,  and 

also  select  a  y^eY.    Let  (the  I's  in  the  binary  number)  B  denote  a  set  of  —  higher 

order  bit  positions,  which  are  not  input  during  t,  for  members  of  the  X-pool.  Let 
m  be  the  number  of  K^cY,  which  have  the  same  bit  values  as  Yj  off  of  B,  and  for 
which  s;i  =  Sj;  m  =  |{y^eY  |  Yj^^B^Yj^B,  s;^  =  Sj}\.  li  m<2\  the  process  is  repeated 
with  another  Y  selection.  Otherwise,  the  matching  K's  are  removed  from  Y  in 
groups  of  2%  and  corresponding  X's  are  removed  from  the  pool.  Each  X  is 
assigned  the  k  —  s  leading  bits  of  its  Y.  The  lower  order  bits  of  a  2'-member  X- 
group  are  assigned  some  permutation  of  the  respective  5-bit  values  0,1,..., 2^  —  1. 
If  the  number  of  matching  Y's  is  not  a  multiple  of  2%  it  suffices  to  leave  that  last 
remnant  Y  group  in  Y. 

When  an  X  pool  has  fewer  than  2^  remaining  members,  it  is  discarded  and  a 
new  pool  (of  size  s  2^)  is  started  along  with  new  selections  of  y's  from  Y.  Since 
we  have  assumed  that  the  number  of  X  pools  is  sufficient,  the  process  will  only 
terminate  when  no  KeY,  for  the  selected  X,  gives  a  sufficient  count  for  m.  In 
that  case,  the  total  number  of  y's  remaining  in  Y  is  small.  For  each  value  s,  X 
partitions  the  set  {y^cY  :  Si^  =  s}  into  2^^"^"*^^  equivalence  classes.  Evidently  each 
class  contains  fewer  than  2^  y's.  Thus  the  number  of  lower  order  output  bits, 
which  are  output  during  t,  and  which  belong  to  the  y's  left  in  Y,  is  bounded  by 


>>-•'';! 
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^27*-— ^,<   2^=^22/^2/3 


—  -,1/3 

s=l  ^ 

K  we  require /2^<—'n''^,  then  at  least  "0^  output  bits  will  be  assigned  values 
by  this  process.    The  other  constraint  on  /  is  that  the  pool  size  be  large  enough: 

2/<-l-„08^92     -j^^s^  it  is  sufficient  to  set  €-.07,  so  that  — <2*<(— )^  '^  (and 
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have  r>8  or  /I  large). 

The  accounting  for  the  excess  pools  of  X's  is  now  simple.  For  each  of  the  16 
multiples  of  a  pool,  keep  the  one  whose  corresponding  Y  set  has  the  maximum 
number  of  lower  order  bits  that  are  output  during  t.  Let  the  set  V  comprise  those 
Y  collections.  Thus,  the  K's  in  V  have  at  least  T\f/\92  lower  order  output  bits 
which  can  have  various  values,  depending  on  the  unread  X  bits,  and  which  are 
output  during  t.  All  remaining  y's  are  paired  with  unmatched  (genuine)  X's, 
which  are  assigned  the  value  of  their  corresponding  y's. 

Step  5.  By  construction,  V=UZx,  where  the  union  is  over  disjoint  sets,  and 
each  Z  contains  (for  some  s)  exactly  2'  of  the  }'*'s,  each  of  which  has  s*  =  s. 
Moreover,  those  Y's  can  have  their  lower  order  bits  assigned  any  permutation  of 
the  5-bit  numbers  0,1,..., 2^-1,  depending  on  imread  bit  values  of  the  X's.  Let  b^ 
be  the  number  of  lower  order  bits  that  belong  to  the  K's  in  Zx,  and  that  are  out- 
put during  i:  b\  =    ^  l>^yl-    ^^  ^x  ^  the  number  of  different  possible  output 

configurations  (assignments)  for  the  ^x  bits.  Clearly  ^x  c^n  be  much  less  than 
2^!,  since  only  Z?x  of  the  ^2'  lower  order  bits  belonging  to  the  KeZx  are  output 
during  t.  More  precisely^, 

ATx  >  -^  >  ^^^  =  e-^'2'^  >  2'^  -  ^■^^^. 

Therefore,  if  each  KeZx  has,  on  the  average,  at  least  2  output  bits,  so  that 
bx  ^  2'^^  then  bx^.25bx  + 1.5x2',  whence  Nx>2-^''\  If  ^x<2'^^  then  an  ade- 
quate estimate  for  A^x  is  based  on  the  fact  that  exactly  2'"^  of  the  5'th  least  signi- 
ficant bits  will  be  I's: 

A^x  ^  (2^-1)  ^  22'-'>2-^\        bx<T^K 

It  follows  that  the  total  number  of  possible  lower  order  outputs  produced  during 

T  due  to  the  uncertainty  in  the  unread  X  bits  is  at  least  f|iVx  —  2"^^^ ,  and  hence 

A  =  n(-log-^^^^ ).  D 


^re     we     make     mild     use     of     a     "sn-ong"     form     of     Stirling's     Formula     [Q     pp.     504-507]: 
(2-n)  Ve-"<n!<(l  +  y4n)(277n)  Ve"". 
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A  few  definitions  will  help  simplify  the  subsequent  exposition.  Given  n  k-hit 
variables,  X={X,}f=i  and  an  n-tuple  11  =  (xj,  X2,...yX„)  of  /-bit  numbers,  where 
/<jk,  we  say  0  is  a  higher  (tower)  order  assignment  to  X  if,  for  i-l,2,...,n,  the 
number  comprising  the  /  more  (less)  significant  bits  of  X,  equals  x,.  Given  a  mul- 
tiset 2  of  n  /-bit  numbers,  where  /<it,  we  say  X  has  a  higher  (lower)  order  restric- 
tion to  2,  if  the  higher  (lower)  order  assignment  to  X  is  an  n-tuple  comprising  the 
members  of  2.  If  5  is  a  set  of  n-tuples,  X  is  restricted  to  S  means  the  X's  can  be 
assigned  any  tuple  in  S.  Given  a  subset  x  of  the  bit  variables  in  {Xjf^i,  an 
assigimient  to  x  is  an  assignment  of  values  to  x-    If  if  is  an  assignment  to  x,  and 

S  is  a  multiset  of  n  I  bit  numbers,  then  —  is  the  collection  of  X-assignments  (n- 

tuples)  comprised  from  2  and  agreeing  with  R  on  x-  Fmally,  the  symbol  c  is 
used  to  denote  a  kind  of  tuple  concatenation:  if  S  =  {(a,b,c),(d,eJ)},  and 
U=(x,y,z),  then  ScU  =  {(ax,by,cz),(dx,eyjz)},  where  ax  means  bit  string  a  con- 
catenated with  bit  string  x. 

The  proof  of  Theorem  2,  with  r  set  to  1,  shows 

Corollary  1. 

Suppose  n<2''  <  n^.   Let  x  be  a  set  of   "  °^    input  bit  variables,  each 

belonging  to  the  logn  more  significant  positions,  and  let  ^  be  a  set  of  output  bit 
variables   which    have   significance    1  +  logn    or   lower.     Suppose   further,    that 

2* 
|^|  =  ft(nlog — ).    Then  there  is  a  multiset,  U,  of  n  (logn)-bit  numbers,  a  lower 
n 

order  assignment,  L,  of  n  (*-logn)-bit  numbers  to  X,  and  an  assignment,  R,  of 
values  to  x  so  that 

/(  ^I'  I  (Xi.  .  .  .  .X„)  ^JCL)  =  il(\^\). 

Proof:  It  is  evidently  sufficient  to  restrict  |^|  to  bits  comprising  at  most  .OTlog/i 
different  lower  order  significances,  since  a  good  choice  of  the  lower  order  signifi- 
cances will  only  reduce  |^|  by  at  most  a  factor  of  .07.  The  construction  is  now 
immediate.  D 

For  completeness,  we  observe  that  the  proof  of  Theorem  2  shows  a  little 
more. 

Corollary  2. 

Suppose  n<2*  <  n^.   Let  x   t>e  a  set  of    "  ^    input  bit  variables,  each 

belonging  to  the  logn  more  significant  positions,  and  let  ^  be  a  set  of  output  bit 
variables  which  have  significance  1  +  logn  or  lower.  Suppose  further,  that 
l^l^/i'^.  Then  there  is  a  multiset,  U,  of  n  (logn)-bit  numbers,  a  lower  order 
assignment,  L,  oi  n  (*-logn)-bit  numbers  to  X,  and  an  assignment,  R,  of  values 
to  X  so  that 

/( ^\(x„...  .x„)  ^jcL)  =  n(m). 
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Proof:  As  in  Corollary  1,  it  suffices  to  restrict  the  number  of  different  bit  signifi- 
cances comprising  I^Cl.  Since  k  is  large,  we  set  the  logn  more  significant  bits  of  Yj 
to  the  value  7-1.  Fix  i:  =  ife-logn,  and  let  Y={Yi.  Y2.  ■  ■  ,K„}  in  the  partitioning 
scheme  of  Theorem  2.  Only  y's  which  have  output  bits  in  ^  are  selected  from 
Y,  although  the  "matching"  y's  need  not  have  any  outputs  during  t.  In  this  case, 
1^1  is  so  small  that  one  pool  of  X's  will  suffice  for  all  selected  K's;  no  output  bits 
in  ^  will  be  "wasted.  "D 

The  next  theorem  is  actually  a  lemma.  Formally,  it  extends  the  results  of 

2^ 
Corollaries  1  and  2  to  the  full  range  |^|<nlog — .    Its  principal  purpose,  how- 

n 

ever,  is  to  strengthen  the  area  bound  for  the  case  (— )^<A:<n^(^\ 
Theorem  3. 

Suppose  n<2*  ^  rr.   Let  x   be  a  set  of  — -=—   input  bit  variables,  each 

belonging  to  the  logn  more  significant  positions,  and  let  ^  be  a  set  of  output  bit 
variables,  which  is  contained  in  {7,  /  |  n/3^/<2n/3,  logn</}.  Then  there  is  a  mul- 
tiset, JJ ,  of  n  (logn)-bit  numbers,  a  lower  order  assignment,  L,  of  n  (ik-logn)-bit 
numbers  to  X,  and  an  assignment,  R,  of  values  to  x  so  that 

/( ^  I  (Xi, . . .  .xj  €  f  oL )  =  n(l^l). 

Proof:  Note  that  the  output  bits  ^  belong  to  nimibers  ranking  in  the  middle  third 
of  the  n  values.  Furthermore,  they  are  of  sufficiently  low  significance  that  they 
need  not  affect  the  ordering  of  the  y's. 

A  fooling  set  construction  is  as  follows.    We  rename  the  X's,  if  necessary,  so 

that  Xx,X2,  .  .  .  ,X^3  have  at  most  — -^  bits  in  x-  Let  xi  name  that  subset  of 

o 

bits.    We  reserve  input  variables  Xx^miZ'  ^i-^nn»     ■  ■  >^n  to  have  values  0  or 

2*-L    Xj  though  X„/3  will  have  a  higher  order  restriction  to  2  =  {l,2,...,n/3}. 

We  call  the  the  higher  order  bits  of  Xi  through  X^^  the  permutation  bits. 

Since  at  most  — 7°—  of  the  permutation  bits  are  assigned,  the  number  of  pos- 
sible permutations  due  to  the  uncertainty  in  the  remaining  bits  is  (on  the  average) 

3  ■ 
at  least .    So  it  is  possible  to  find  a  suitable  assignment  /?  on  xi  so  that  at 

2^ 

iL, 

2  3 " 

least  this  number  is  attained.   Let  $  =  — .   We  have  ensured  that  \^\^ 


2^ 


The  next  issue  to  resolve  is  the  number  of  X  variables  which  are  likely  to 
have  many  values,  based  on  selections  from  $.  Suppose  we  consider  two  permu- 
tations (n-tuples),  chosen  at  random  from  <^.    TTie  probability  that  the  two 
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permutations  are  the  same  is  at  most  ,  which  is  quite  small.    In  fact,  the 

3' 
probability  that  the  permutations  differ  for  fewer  than  3  X's  is  less  than 


P  ^     _n  ,  (/i/3  -  p)!  ' 

3' 

It  follows  that  for  n  large,  the  probability  that  3<.ln  is  less  than  . 

(.u5n)! 

X's  lower  order  assignment,  L  is  chosen  at  random,  with  0  and  1  weighted 

equally. 

We  would  like  to  conclude  that  for  some  choice  of  L ,  the  number  of  distinct 
assignments  to  ^  is  large,  when  the  X's  are  restricted  to  ^cL.  We  might  com- 
pute a  lower  bound  for  this  number  by  running  the  following  experiment.  Pick 
two  permutations  in  4>  at  random,  and  estimate  the  probability  P  that  the  result- 
ing assignments  to  ^  are  identical  for  the  two  cases.  Clearly  P  <  F+S,  where  F 
is  the  probability  that  fewer  than  .In  X's  changed  their  rankings,  and  S  is  the  pro- 
bability that  at  least  .In  rankings  changed  but  the  values  assigned  to  ^  were  the 
same  in  both  instances.    Evidently,  if  P  is  small,  then  <t>  must  be  large. 

The  obstacle  in  computing  5  is  that  those  X's  which  switched  positions  might 
not  have  any  output  contributions  to  ^.  We  would  like  to  assert  that  every 
midrange  Y  is  equally  likely  to  be  assigned  many  different  X's,  but  this  may  not 
be  true.  The  difficulty  is  circumvented  by  changing  the  experiment  slightly  to 
include  an  averaging  process.  We  pick  two  permutations  and  estimate  the  proba- 
bility P,  that  the  output  is  pairwise  identical  for  each  of  2n/3  variants.  The  vari- 
ants differ  only  in  the  assigrmaents  to  the  reserve  of  2n/3  X- variables.  Variant  j, 
for  y  =  l,2,...,2n/3,  has  j  of  those  variables  variables  assigned  0,  and  2n/3  -  j 
assigned  2*-l. 

Put  P^F+S,  with  the  obvious  meaning  for  5.  Our  experiment  thus  picks 
two  permutations  at  random  from  O,  and  generates,  for  each  K,,  a  vector  of  2n/3 
X-variables  assigned  to  F,.  If  the  permutation  pair  has  p  X-variables  with  dif- 
ferent rankings,  then  for  any  y,,  where  n/3^i<2n/3,  the  two  corresponding  vec- 
tors will  have  different  X-assignments  for  p  coordinates  (i.e.  variants). 

Let  Yh  have  a;,  lower  order  bits  belonging  to  ^,  for  n/3</i<2n/3.  Suppose 
y,,  for  variant  j,  is  assigned  two  different  X  input  variables.  Then  by  indepen- 
dence of  the  lower  order  bits,  the  probability  is  2  "'  that  for  variant  j,  the  bits  of 
y,  output  during  t  are  identical  for  the  two  permutations.  Permutations,  how- 
ever, are  not  fully  independent  transformations.  In  the  worst  case,  our  permuta- 
tion will  be  comprised  entirely  of  2-cycles.  Suppose,  for  example,  that  the  permu- 
tation pair  causes  X,  and  Xf,  to  exchange  rankings  r  and  s  for  variant  j.  Then  the 
probability   is   at   most  2"™"^""  "^)<2~-^('''+"'^  that   the  output,   for   the   two 
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permutations,  is  identical  on  the  lower  order  bits  which  are  output  during  t,  and 
which  belong  to  Y^  and  Y^  for  variant  j.  Thus  in  the  worst  case,  the  probabihty, 
Sj,  that  the  two  experiments  result  in  identical  outputs  for  variant  j  is  at  most 

2  '  ,  where  the  sum  is  taken  over  Ij,  the  set  of  Y  indices  having  different 
assignments  for  variant  j.  Since       2      2"i  ~  P2"i'  '^  follows  that  for  some 

l<;^2fl/3  id,  i 

3  4/1/3  ^ 

variant  h,  ^a.  ^  2^/3  ^""  ^^^°^  '^''-^  '     • 

Since  S^Sf,,  we  set  ps.ln,  and  use  the  fact  that  y«i=  l^k  to  conclude  that 

_3M 

5^2     ''^  .   A  more  formal  proof  of  this  fact  might  read, 

_iM  _3M 

5^£[£[min(5,),  3>.ln|3]]<£[£[2     "^    |  P]]  =  2     '^. 

h 

Let  V  be  the  number  of  distinct  output  configurations.  More  precisely,  v  is  a  ran- 
dom variable.  It  counts,  for  each  possible  assignment  of  lower  order  bits  L,  the 
number  of  distinct  output  configurations  in  {0,l}^°''x{l,2,...,2n/3}  that  can  be 
assigned  to  ^,  when  X  is  restricted  to  4>oL.  Let  8x  be  the  number  of  different 
permutations  giving  the  X'th  output  configuration,  for  X  =  l,2,...,v,  and  some 

enumeration  of  actual  output  configurations.    Put  8  =  2Sx»  so  that  8  =  |4>|.    The 

1 
8x,  for  X^v,  are  positive  random  variables;  they  depend  on  L.   Now  it  is  simple 

to  verify  that^ 

E[±{^y-]  ^  E[v\\  =  £[^]. 


And  by  definition, 


p  -  E[±(^n 


1  " 

Furthermore,  we  have  already  established  that 


3W 


Prob{{eweT  than  .In  X's  have  different  rankings} +  2     **">/». 

So    TTTTT  ^ ThiH-  =  2"("^{l^l-  "'"g^^  =  2"^l"^l).     It    follows    that 

£[l/vj  .,  -^1^1 

+2     "^ 

there  must  be  an  assignment  to  the  lower  order  bits  for  which  v  =  2^(l'^l^.    Evi- 
dently each  distinct  output  configuration  must  have  a  corresponding  permutation 

■'Alternatively,  this  is  just  a  special  case  of  Jensen's  Inequality  [D  p.  16]:  If  j  is  a  convex  funaion,  then 
~'^gi~r)-gi~'^~r)-  Setting  g(z)  =  v:^  and  taking  the  expeaed  value  gives  the  inequality. 
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2n/3 
belonging  to  <I).    Thus  v^2^i»  where  v,  is  the  number  of  distinct  outputs  for 

1 

variant  i.  Hence  there  must  be  at  least  permutations  giving  distinct  outputs 

for  one  of  the  variants,  and  some  L.  Taking  the  logarithm  gives 
/  =  ft(l^l)  -  logn.  We  note  that  if  |^|  =  0(logn),  then  Corollary  2  suffices  to 

show/  =  n(|^|).n 

We  remark  that  the  proof  technique  of  Theorem  4  can  also  be  used  to  show 
/  =  n(|^I'|)  when  |^|  =  0(logn). 

Curiously,  Theorem  3  appeairs  to  be  wejik.    The  obvious  application  is  to 
combine  it  with  the  fact  that  there  must  be  a  suitable  interval  of  time  during 

which  at  most  — r°—  more  significant  bits  are  input,  and  during  which  n —    °^ 
2  or 

suitable    lower    order    bits,    ^,    are   output.     The   conclusion    would   be   that 

A  =  n(— log ),  which  is  a  bit  weak  (the  best  bound  is  n(— log )  ),  and 

r  n  r  n 

does  not  hold  for  the  desired  range  of  it. 
Theorem  4. 

Suppose  each  input  is  read  r  times,  and  (— )^<2*^n.   Then  A  =  ft( — ). 

2k 
Proof:  Put  K=— -,  71  =  2'*.  We  will  apply  Theorem  3  to  a  sorting  problem  having 

T]  rs  and  X's. 

Standard  argvmients  show  that  there  is  an  interval  of  time  t,  when  no  more 

Im 

than  —  bits  are  read,  which  belong  to  the  k  more  significant  positions,  and  when 

kn 
——  bits  are  output,  which  belong  to  the  outputs  7,  /,  where  n/?>-^i<2n/?>,  and 

/>K.    Let  the  y's  having  these  lower  order  output  bits  comprise  {l'x}xt\-    Clearly 

A  A 

kn       k2  ^  k2  ^2"*  n 

\A\^——<——.    Furthermore,  it  is  easy  to  check  that  <^i-.    So  |A|<^. 

lor       lo  18        3  3 

Let  Ai  =  {1,2,...,1(t,-|A|)/2J},  and  A^={f-^  +  l,  .  .  .  ,^+ [(t,-|A|)/21-1}. 

J,      J  J  _  I A  I 

Put  A  =  AiUAUA2.    It  suffices  to  assume  that  n>3,  whence  — >-^>-Hr' — '-. 

This  inequaUty  ensures  that  the  indices  in  A  are  between  the  indices  of  A^  and 
A2.  Clearly  |A|  =  t).    We  rename  the  X's,  if  necessary,  so  that  Xj,  X2,  ■  ■  ■  ,X^ 

Tl  K 

have  at  most  -j—  bits  which  are  input  during  t,  and  which  belong  to  the  k  more 

significant  positions.  This  is  possible  since  on  the  average,  each  X  has  only  — 
higher  order  X-bits  read  during  t.  We  apply  Theorem  3  to  this  reduced  problem, 
and  conclude  that  A  =ft( — )  for  sorting  {X,}-Li  into  {J'^IxcA- 


rjl;   •■.  - 
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To  complete  the  proof,  we  must  ensure  that  the  larger  problem  contains  our 
specific  T)- number  sorting  problem.  This  is  easy.  It  suffices  to  give  the  remaining 
inputs,  {X,}f=T,+i  fixed  values,  assigned  in  batches.  Let  A  =  {X,}/Li,  where 
Xi<X2<  •  •  •  <>^TT  Put  Xo=0-  Let  n-\^  X's  be  2*-l,  and,  for  i=l,2,...,r\,  set 
X,-X,_i-1  X's  equal  to  ^,.  Define  the  ^'s  so  that  ^,  /=0  for  />k,  and  ^,  /=l'x  / 
for  /^K.  It  is  easy  to  see  that  these  X's  have  the  right  rankings;  they  fill  the  Y 
gaps  correctly.  IH 

It  should  be  observed  that  in  Theorem  4,  we  construct  a  permutation  on  a 
thin  set  (|^|«/i).  Theorem  2  fails  for  such  a  case  (unless  |  PSI  |  is  very  small) 
because  the  output  bits  may  be  mostly  "wasted."  Note,  too,  that  both  Theorem  2 
and  Theorem  3  are  in  fact  quite  different,  although  both  show  that  there  are  bit 
collections  having  high  information  content  despite  the  presence  of  constraints. 
Theorem  2  asserts  the  existence  of  a  set  of  lower  order  output  bits  with  high 
information  content  (even  when  r  is  large),  while  Theorem  4  asserts  that  every 
collection  of  lower  order  output  bits  (taken  from  the  midrange  J"s)  has  high  con- 
tent. 

This  distinction  is  fundamental.  In  Theorem  2,  the  number  of  lower  order 
bits  was  high,  but  k  could  be  so  small  that  many  of  the  lower  order  bit  ensembles 
might  not  jointly  have  high  content;  there  might  not  be  enough  higher  order  signi- 
ficant bits  to  guarantee  independence.  However,  once  a  suitable  lower  order  set 
was  chosen,  (legal)  permutations  could  be  found  to  adequately  "mix  up"  many  of 
its  bit  assignments.  No  y's  without  timely  lower  order  output  bits  were  essential 
to  the  permutation  construction.  The  difficulty  with  using  these  "empty  y's"  is  to 
make  certain  that  they  are  not  overcommitted,  that  is,  to  ensure  their  lower  order 
assignments  are  not  needed  by  members  of  ^  assigned  to  different  X  "pools". 
This  difficulty,  of  course,  vanishes  for  |^|  =  0(n^),  since  then  then  one  pool  can 
be  used  for  all  essential  y's.  Since  |^|«n,  for  the  one  application  of  Theorem  2, 
such  an  over  commitment  would  seem  unlikely  in  this  case.  This  thinness  of  |^| 
is  what  motivated  the  choice  of  a  probabilistic  proof  technique  in  Theorem  3.  The 
result,  of  course,  turns  out  to  hold  for  larger  sets  of  output  bits  as  well. 

Theorem  2  and  Theorem  4  can  be  combined  to  read. 

Corollary  3. 

Suppose  r<n,  and  -<2*<n^(^).   Then  A=ft(-log-^^^ ).  D 

r  r  n 

The  following  lemma  improves  our  lower  bounds  when  r  is  large. 

Lemma  1. 

For  any  number  of  replicated  inputs,  the  area  of  a  circuit  that  sorts  n  it-bit 
numbers  must  satisfy,  A  =  ft(logn). 

Proof:  The  reason  for  this  bound  is  that  a  circuit  that  inputs  mk  bits  and  outputs 
nk  must  have  enough  internal  states  to  be  able  to  count,  effectively,  to  min(r,n). 
Adding  logr,  for  r<n,  to  the  bounds  in  Theorem  1  or  Corollary  3  is  equivalent  to 
adding  ft(logn).n 
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3.  AT^  Bounds 

When  r=  1,  both  Theorem  2  and  Theorem  3  show 
Theorem  5. 

Suppose  n^2'^^n^^^\    Then  a  when-  and  where- determinate  circuit  that 

sorts     n      k-bit     numbers      and      reads      the     inputs     once      must     satisfy 

9/fc  +  l 

AT^=ilin^~\og^- ). 

n 

Proof:  We  may  assume  k^l  +  \ogn,  as  otherwise  we  can  replace  n  with  n/2.    We 

_        2^ 
can  also  assume  7<— log — ,  as  otherwise  the  result  is  tri/ially  true.   The  chip  is 

3         n 

partitioned  in  the  standard  manner  so  that  each  side  of  the  circuit  outputs  at  least 

n       2^ 

—log —  bits,  which  are  of  significance  l  +  logn  or  below.  At  least  half  of  the 

3        n 

more  significant  bits  (i.e.,  of  logn  significance  or  better)  must  be  input  on  one 

side  of  the  circuit,  say,  the  left.  Then  Theorem  3  and  the  proof  of  Theorem  2 

2* 
both  show  that  the  right  side  needs  n(nlog — )  bits  of  information  from  the  left, 

n 

if  the  circuit  is  to  sort  correctly.  D 

We  remark  that  the  proof  technique  used  for  Theorem  5  cannot  give  a  better 

AT^  lower  bound,  because  it  is  based  on  estimating  the  total  information  flowing 

across  a  single  partition  of  the  input  ports.  If  the  data  is  partitioned  into  two  sets 

of  numbers  5^  and  ^2,  then  the  requisite  information  flow  caimot  be  proven  to 

exceed  the  information  content  of  the  sorted  sets.   As  shown  in  [S],  n  k-bit 

2* 
numbers  can  be  easily  represented  in  0(nlog — )  bits. 

n 

For  completeness,  we  note  that  in  [S]  is  proved 
Theorem  6  [S]. 

Suppose  2*^n.  Then  a  when-  and  where-determinate  circuit  that  sorts  n  it- 
bit  numbers  and  reads  the  inputs  once  must  satisfy  AT^=Cl(2^log^—j—r).n 

4.  Upper  Bounds 

Circuits  for  sorting  n  ife-bit  numbers  were  presented  in  [S].  Theorems  7  and  8 
are  simple  applications  of  the  sorting  schemes  given  there.  The  only  difference  is 
that  since  the  inputs  can  be  read  r  times,  it  suffices  to  use  the  sorting  schemes  to 
output,  for  input  phase 7,  Y^+j^^.  Yj+jn/r.  ■  ■  ■  .Y^r+jn/r,  for  j=0,l,...,r-l.  Such 
a  circuit  requires  sufficient  size  to  contain  pass  fs  intended  output  of  n/r  sorted 
*-bit  numbers,  the  (largest)  X-index  of  the  maximum  value  so  stored,  a  counter 
possibly  going  up  to  n,  a  small  amount  of  controller  logic,  and,  say,  two  tem- 
porary registers  of  k-  and  logn-bits.  As  a  consequence  of  the  data  storage 
schemes  in  [S],  it  follows  that 
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Theorem  7. 

For  input  replicated  r  times,  a  set  of  n  it-bit  numbers,  where  2*^<  — ,  can  be 

sorted  in  area  A  =  0(2*log — -^  +  logn). 

This  result  is  clearly  optimal  for  all  values  of  r. 
Theorem  8. 

For  input  replicated  r^n  times,  a  set  of  n  ik-bit  numbers,  — <2*^n^,  can  be 

sorted  in  area  i4  =  0(— log H  logn). 

r  n 

This  result  is  also  optimal. 

The  next  sorting  construction  is  optimal  for  some  values  of  r  and  k,  and 
provably  suboptimal  for  others. 

Theorem  9. 

For  input  replicated  r<n  times,  a  set  of  n  Jt-bit  numbers,  where  2*>n-,  can 
be  sorted  in  area 

A  =  0(-logn)  +  0(G), 

where  G  =  min  {n,  — ,    — r-logn}. 

r       r^ 

Discussion:  For  r<logn,  n  is  dominated  by  —logn,  and  hence  the  area  bound  is 

tight  for  r  =  C>(logn).  The  first  term  in  the  expression  for  G  comes  from  using  n 
input  pads,  one  for  each  number,  and  marking  which  inputs  have  been  already 
sorted.   If  other  pad  markings  indicate  which  inputs  are  candidate  maximum 

values  for  the  current  —  group  of  values  currently  being  sorted,  and  which  are 

evidently  smaller  values  in  the  group,  then  only  0{n)  bits  are  needed  in  addition 

to  the  number  necessary  for  a  rank  and  forward  scheme  on  —  numbers.  The 

second  term  arises  from  the  obvious  approach  of  storing  the  next  —  numbers  to 


sort.    The  third  results  from  intermediate  passes  to  locate  the  addresses  (indices) 
of  the  current  — r  numbers  to  sort.  In  this  case,  all  pairs  of  numbers  are  com- 

pared  in  each  of  r ^  sorting  phases,  and  the  addresses  of  suitably  ranked  values 
are  stored. 

A  natural  question  arises  as  to  the  why  the  gap  occurs  for  sorting  with 
repeated  inputs  and  large  k.  Part  of  the  the  discrepancy  is  caused  by  the  format  of 
the  output  schedule,  and  the  failure  of  our  techniques  to  captiire  and  to  quantify 
adequately  two  constraints  inherent  to  the  format.  U  output  were  not  required  to 
be  when  determinate,  but  instead  were  required  to  comprise  ordered  pairs  (i,x), 
where,  /  is  the  rank  of  input  value  x,  then  the  solution  would  be  meaningful,  and 
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the  circuit  would  certainly  be  a  sorter.  It  is  not  difficult  to  construct  a  circuit 
which  sorts  in  this  manner,  reads  the  input  r  times,  and  which  uses  an  area  pro- 
portional to  — logn  area.  Furthermore,  if  such  sorted  data  were  then  used  as 

input,  another  kind  of  sorter  could  be  designed  that  reads  its  data  r  times,  pro- 
duces the  standard  where-   and  when- determinate  output,   and  that  has  area 

A  =  — logn.    Both  tasks  are  easy.  The  obvious  composition,  however,  takes  r^ 

input  passes  for  area  —logn.    The  result  is  an  alternative  construction  giving  the 

— rlogn  term  for  G  in  Theorem  9. 

Also,  the  information  theoretic  nature  of  the  proofs  ensures  that  for  repli- 
cated inputs,  the  lower  bounds  arguments  would  hold  if  an  oracle  were  consulted 

r  times.  A  good  oracle,  for  example,  might  return  the  addresses  of  the  next  — 
numbers  to  output,  whence  the  requisite  area  would  be  0(— logn).  This  bound 

accounts  for  an  0(— log— )  rank  and  forwarding  cost  plus  an  0(— logr)  area  to 

store  the  addresses.  Evidently,  a  more  deUcate  decomjxxition  of  time  and  infor- 
mation flow  is  needed  to  establish  better  lower  bounds  in  this  case. 

We  now  describe  a  sorting  scheme  that  reduces  the  bounds  gap  for  some 
cases. 

4.1.   Partitioning  techniques  for  sorting 

In  [S],  a  rank  and  forward  sorting  scheme  is  shown  to  work  well  when  the 
number  of  bits  is  large,  say,  for  ik>21ogn.  An  essential  aspect  of  the  scheme, 
however,  is  to  be  able  to  input  all  the  numbers  in  bitwise  parallel.  The  purpose 
of  this  section  is  to  address  the  problem  of  rank  and  forward  schemes  when  the 
area  is  less  than  n.  As  in  the  previous  Section,  the  sorting  will  proceed  on  con- 
secutively ranking  values.  The  problem  is  how  to  determine  which  inputs  to  sort 
in  a  given  pass,  so  that  rank  and  forward  sorting  can  be  applied  to  the  selected 
variables. 

Our  rank  and  forward  sorting  for  large  r<n  is  based  on  partitioning.  We 
suppose  the  scheme  sorts  x  numbers  per  phase.  Evidently,  it  suffices  to  know  the 
addresses  of  the  x  numbers  for  the  current  phase.  The  problem  is  how  to  identify 
them,  and  how  to  remember  them.  We  describe  methods  which  store  the 
addresses  of  the  numbers  to  be  sorted  in  the  current  phase.  In  view  of  Theorem 
9,  these  addresses  can  be  stored  without  appreciably  increasing  the  area  required 
to  sort;  the  real  difficulty  is  how  to  identify  the  numbers.  One  way  to  identify 
them  is  by  ranking  all  n  numbers  during  each  phase.  The  addresses  are  likely  to 
require  storage  even  for  this  naive  sorting  scheme  if  the  output  is  to  be  when- 
determinate,  since  specific  rankings  might  be  discovered  at  an  indeterminate  time, 
when  not  all  the  bits  are  available  or  are  scheduled  to  be  output.    This  method 
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can  be  implemented  by  comparing  all  pairs  of  numbers.  One  possible  implemen- 
tation is  outlined  in  the  concluding  remarks  of  Section  2. 

We  now  describe  sorting  algorithms  which  are  better,  at  least  when  k  is  not 
too  large.  They  will,  however,  require  ft(Jfc)  storage.  Since  the  methods  are 
somewhat  compUcated,  inferior  algorithms  will  be  presented  first,  and  modifica- 
tions will  be  subsequently  outlined  and  analyzed. 

In  the  following  discussion,  it  is  convenient  to  assume  that  all  input  values 
are  distinct.  This  assumption  causes  no  difficulty,  since  the  sorting  concerns  the 
case  when  it>21ogn,  and  we  can  then  use  lexicographical  sorting  with  the  index 
(address)  of  each  input  variable  as  the  secondary  key. 

Evidently  the  relevant  addresses  can  be  computed  in  one  pass  if  the  the  larg- 
est and  smallest  values  in  the  current  sorting  phase  are  known. 

A  reasonable  scheme  to  sort  x  numbers  per  pass  might  be  to  initially  store 

the  addresses  of  the  h'th  largest  numbers,  for  h  =  x,2x,4x,8x, We  call  such  a 

sequence  an  ordered  partitioning  branch.  These  values  could  be  found  via  tech- 
niques to  compute  order  statistics.  Then  only  a  small  amount  of  computation 
might  be  necessary  to  update  the  sequence  for  each  new  sorting  phase. 

It  turns  out  that  this  scheme  can  be  implemented.  Improvements  are  based 
on  modifying  the  standard  order  statistic  algorithms,  relaxing  the  requirement 
that  they  find  exact  order  statistics,  and  relaxing  the  requirement  that  the  rank- 
ings of  the  elements  (represented)  in  the  ordered  partitioning  branch  be  exact. 

Given  a  set  S,  define  5^^  to  be  {5e5[x<j^y}.  Given  a  finite  set  S  with 
\S\  =  c,  we  call  a  sequence  -^^sq<si<S2<  ■  •  •  <sj<Sj+^  =  oo,  an  x-k- 
decomposition  of  S  if 

(1)  5o=  -°°,  if  ^  =  0,  and  sq  is  the  fcc'th  smallest  member  of  5,  if  k>Q. 

(2)  {5i,...j^}CS. 

(3)  3x>^y,„,J>x. 

(4)  ForJ  =  l,...,rf,  c2-'>|S,^_^,J>c2-'-limpHesc2-'+^>|5,^,J>c2-'-^ 

(5)  For  i  =  0,l,...,J-2,  c2-'>|5,,  sj>c2-'-'  implies  \S,^^^,J>cl-K 

The  addresses  of  the  values  sq,  .  .  .  ,5^^  will  form  our  (generalized)  x-k- 
ordered  partitioning  branch  of  S.  The  requirements  1  through  5  have  simple 
interpretations.  They  are 

(1)  Sq  is  the  largest  value  sorted  so  far  (or  -«»,  if  none  have  been  sorted). 

(2)  The  values  are  in  5. 

(3)  The  partition  of  S  with  the  smallest  values  contains  enough  values  for  the 
current  output  phase,  and  not  too  many  more. 

(4)  If  we  call  the  half  open  intervals  (c2~'~^  c2~']  slot  /,  then  the  sizes  of  two 
consecutive  partitions  in  the  ordered  partitioning  of  S  belong  to  the  same  or 
consecutive  slots. 
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(5)   The  sizes  of  at  most  three  (consecutive)  partitions  of  S  can  belong  to  a  single 
slot. 

We  remark  that  it  will  soon  be  convenient  to  relax  the  defmition  of  an  x-k- 

ordered  partitioning  to  allow  some  empty  slots.  The  reason  for  this  next  (and 

last)  modification  is  to  define  an  invariant  that  allows  an  ordered  partitioning  to 

be  updated  in  each  sorting  phase  with  only  one  application  of  a  subroutine  call. 

The  subroutine  will,  given  addresses  of  partition  values  5/  and  5/+i,  retiim  an 

2  1 

address  of  a  value  s,  where  si<s<si+i,  and  jK,  ji^J^l^^^^^J^ Y^^^'^'-il-  ^"^ 

we  need  a  separator  algorithm  which  partitions  a  set  into  two  pieces,  and  the 
larger  piece  contains  the  larger  values  and  has  a  size  at  most  two  thirds  of  the  ori- 
ginal set. 

We  postpone  a  discussion  of  the  separator  subroutine  to  concentrate  on  the 
other  aspects  of  the  algorithm.  The  separator  is  used,  of  course,  to  spUt  a  subset 
5fl  c  into  two  pieces,  S^  b  ^^^  ^bc-  It  is  simple  to  verify  that  if  5^  ^  occupied  slot 
/—I,  then  there  are  only  three  possibiUties  for  the  new  pieces.  Either  Si,c  occu- 
pies /—I  and  Sa  b  occupies  /,  or  both  occupy  slot  /,  or  Sbc  occupies  slot  /,  and  S^  b 
occupies  /+1.  The  point  of  this  organization  will  be  that  in  any  case,  the  form- 
erly empty  slot  /  will  get  a  subset. 

The  sorting  algorithm  is: 

Create  x-0-ordered  partition  branch; 
While  not  done  do 

Sort  X  smallest  numbers  remaining  in  a  partition; 

Call  separation  algorithm  and  fill  lowest  empty  slot  (if  any)  by 
sphtting  the  lowest  partition  in  the  next  IcU-ger  slot 
End. 

We  identify  as  the  bottom  slot  a  pair  of  slots,  namely  the  slot  containing  S,^  ,^ 
and  the  next  larger  slot.  Higher  slots  remain  uniquely  named.  We  are  willing  to 
sort  all  the  members  of  a  single  partition  (having  the  smallest  values)  in  the  bot- 
tom slot,  if  the  partition  contains  x  or  more  members.  Otherwise  we  merge  the 
set  with  the  next  piece  and  sort  that.  Correctness  follows  from  the  following 
invariant.  Between  closest  (internal)  empty  slots  is  a  slot  containing  two  or  more 
subsets.  That  is,  if  slots  j  and  l>j  are  empty  (and  above  the  bottom  slot),  then 
either  there  are  two  subsets  with  sizes  in  slot  h,  for  some  h:j<h<l,  or  all 
members  of  the  partition  have  sizes  belonging  slots  below  j.  A  simple  induction 
argument  will  show  that  this  property  is  preserved.  It  is  clearly  sufficient  to 
guarantee  that  two  (internal)  empty  slots  cannot  be  consecutive,  whence  the 
lowest  empty  slot  can  always  be  filled.  In  particular,  the  bottom  slot  will  never  be 
empty  (until  all  the  numbers  are  sorted). 

Creating  the  initial  ordered  branch  is  also  simple.  The  details  are  omitted. 
We  now  consider  separator  algorithms. 
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Suppose  we  read  /  numbers  at  a  time,  and  use  sorting  circuitry  to  find  the 
median  of  the  input.   The  standard  algorithm  [AHU]  for  /  =  5  reads, 

OrderS(5,  m) 

If  |5|</  Return  (m'th  largest  value  in  5); 
Partition  5  into  |5|//  pieces  of  size  /; 
S  -  U{median  of  each  piece}; 

pi^OrderS(5,  ^); 

Si  ^  {seS\s<ii.}; 
S,  ^{s€S\s  =  ^x}^ 
Sg  -  {seS\s>iJi}; 
mi  *-  |5/|; 
nig  -  |5^|; 

If  rtii^m  RetUTn{OrderS{Si,  m)) 
Else  if  nig^m-mi  Retum(OrderS(Sg,  m-mi)) 
Else  Ket\ira(OrderS{Sg,  m  — nig- mi)); 
End. 

This  method  will  be  modified  as  follows: 

(1)  Approximate  order  statistics  are  found,  rather  than  exact  ones. 

(2)  A  depth  first  iteration  is  used,  instead  of  breadth  first  recursion. 

(3)  Rather  than  return  a  median  from  one  level  to  use  as  a  separator  at  the  next, 
separators  from  the  deepest  level  are  returned  to  the  top  level. 

The  consequences  are 

(1)  A  savings  of  input  passes 

(2)  A  savings  of  input  passes  and  area 

(3)  A  savings  of  input  passes  at  a  cost  of  finding  a  very  weak  (preliminary) 
separator. 

Then  /  can  be  tuned  for  the  best  performance. 

First,  though,  it  should  be  noted  that  the  standard  algorithm  as  given  is 
materially  simplified  when  /  is  chosen  optimally.  The  reason  is  that  if  the  algo- 
rithm finds  medians  of  /  numbers  at  a  time,  then  —  median  addresses  must  be 
stored,  whence  a  good  separator  can  be  found  in  2  passes,  and  the  area  for  this 
portion  of  the  algorithm  is  /+— log/.   If  x  numbers  are  sorted  per  phase,  then  the 

circuit  can  be  implemented  in  area  A  =  C>(ik+xlogn  +  /+— log/)  and  input  passes 

r  =  0(— ).  We  set  /=(nlogn)^,  whence  A  =  0(it+(nlogn)^+— logn). 

One  difficulty  with  the  algorithm  as  outlined  is  that  it  is  not  where-  deter- 
minate. The  difficulty  occurs  at  the  outer  level.  When  only  a  subset  of  the  n 

numbers  are  to  be  processed  (due  to  partitioning),  the  —  medians  found  in  the 
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first  pass  will  not  be  medians  of  unifonnly  sized  subsets  if  the  outer  level  is  to  be 
completed  in  one  pass.  Thus  either  additional  passes  are  needed,  or  weighted 
median  techniques  must  be  used. 

It  turns  out  that  weighted  medians  can  indeed  be  sensibly  used,  and  the  area 
bound  is  affected  by  no  more  than  a  constant  factor.  The  details  are  omitted. 

In  the  modified  depth  first  method,  log/n  groups  of  /  addresses  are  stored, 
one  group  for  each  level  of  depth  in  the  algorithm.  The  following  skeletal 
description  shows  the  organization,  but  omits  critical  details  essential  to  correct- 
ness and  efficient  operation. 

Algorithm  Weaksep 

Procedure  Load(k,d) 

U  d=0,  k  -^  address  of  median  of  /  current  values; 

Load(k,l) 
Else  insert  k  into  addressbucket(d); 
If  addressbucket(d)  is  (possibly)  full  then 

k  -  address  of  median  of  values  represented  in  addressbucket(d) ; 

Load(k,d  +  l); 
Retiun; 
End  Load; 

Repeat 

While  a  group  of  /  inputs  are  left  unread  Do 

Load(k,0); 
Flush  lower  levels  (or  assume  n  =  l'). 
Return  final  address. 
Partition  based  on  value  at  address; 
With  bigger  subpiece  until  suitable  separator  found; 
End  Weaksep. 

It  should  be  noted  that  due  to  when- determinism  of  the  input,  separator 
computations  at  level  d  require  l'^  values  to  be  read,  so  that  all  values  correspond- 
ing to  the  stored  addresses  can  be  compared.  At  the  top  level,  the  separator  is 
only  computed  for  a  subset  of  the  inputs,  which  lie  between  two  partitioning 
values.  Consequently,  a  when  determinate  operation  of  this  scheme  that  uses 
input  passes  efficiently  will  empty  buckets  which  are  not  full.  As  a  consequence, 
the  median  computations  are  weighted.  The  partitioning  values  that  are  found 
are  very  weak  separators.  Indeed,  after  log/n  levels  of  recursion,  a  weak  separa- 
tor is  found  which  partitions  the  original  set  into  two  pieces,  neither  smaller  than 
n  °^  .  It  is  easy  to  see  that  if  the  set  containing  the  median  is  partitioned 
further,  and  the  process  repeated,  then  a  strong  separator,  which  divides  the  set 
into  two  pieces,  none  smaller  than  one  fourth  of  the  original  set,  is  found  in 

(— )  °^  iterations.  Moreover,  a  2/3  —  1/2  separator  can  be  constructed  by  apply- 
ing a  small  number  of  calls  to  a  separator  algorithm  that  partitions  sets  into  pieces 
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of  size  1/4  or  larger. 

If  values  (rather  than  addresses)  were  stored,  Weaksep  could  be  imple- 
mented with  the  input  read  only  once.  When  addresses  £u-e  stored,  a  weak 
separator  can  be  found  in  log^n  passes  over  the  input,  which  correspxands  to  a 

(distributed)   input  pass  for  each  bucket  level.    Hence  0((— )  °^  log^n)  input 

passes  are  needed  per  partitioning  phase. 

Recalling  that  /  numbers  valued  0  to  n  — 1  C5ji  be  stored  in  /log—  area,  and 
that  the  depth  of  the  separator  finding  scheme  is  log^n,  we  conclude  that  the  area 
for  the  separation  operation  is  /log— log^n.  K  x  n  imbers  are  to  be  sorted  in  each 
sorting  phase,  then  their  addresses  can  be  found  in  one  input  pass  if  surrounding 
values  are  known.  The  area  to  store  the  addresses  is  0{x\og—).  The  sorting  can 

be  done  in  one  more  input  pass.  Our  ordered  partition  stores  log—  partitioning 
addresses,  and  is  updated  only  once  per  sorting  phase.  Thus  the  total  number  of 
input  phases  (including  initialization  passes)  is  r=0(n  °^  log//ilog— H — n  °^  log/n) 

1  ^      ^ 

and  the  area  is  A  =  0{k+xlog^  +  lloffi\ogin).    Set  /=2^,  where  8  is  fixed.    Then 

-1+6 

A  =  0(k+- log-n). 

1 


If  r^n*^  where  ct<1   ,  setting  /=—  gives,  A  =  0(k+- logn).    The 

r  1-cT    r 

corresponding  expression  is  a  bit  tamer  for  l=enJr. 
Theorem  10. 

For  input  replicated  r<n*'  times  where  a<l,  a  set  of  n  Jt-bit  numbers  can  be 
sorted  in  area  y4  =  0(— logn  +  k).  D 

Evidently,  if  ik=C>(— logn),  then  such  a  sorting  circuit  can  be  implemented  with 

an  area  that  is  within  a  constant  factor  of  optimal. 

5.   Conclusions 

We  have  shown  that  for  inputs  read  r<n  times,  n  integers  in  the  range 
[0,m  —  1]  can  be  sorted  in  area 


A{n,m,r)  =  0 


logn  +  mlog-=^,         m^n/r, 
rm 

logn  +  -^log^^.       n/r^m<n'^. 


These  bounds  also  apply  to  a  bit  modeled  sorting  network,  since  the  sorters  are 
comprised  of  Uttle  more  than  circular  shift  registers,  counters  and  comparators. 
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These  devices  can  be  built  with  bounded  fan  in  and  fan  out,  and  can  operate  in  a 
systolic  or  (partially)  self  timed  manner,  without  the  benefit  of  global  clock  or 
control  lines. 

For  a  VLSI  circuit  model,  the  techniques  show  that  if  the  inputs  are  read 
once,  then  a  when-  and  where-determinate  circuit  that  sorts  n  integers  in  the 

range  [0,m-l],  for  n^m<n^^^\  must  satisfy  >4r^=fl(n^log^ ).   Moreover,  this 

n 

bound  is  the  best  pxjssible  for  the  proof  technique  used. 

It  is  worth  observing  that  the  proof  of  the  AT^  bound  is  based  on  the  inevit- 
able misallocation  of  the  more  significant  input  bits.  The  distribution  of  lower 
order  input  bits  is  irrelevant.  In  this  sense,  the  standard  proof  for  ik=l-l-logn  is 
somewhat  misleading,  since  it  draws  attention  to  the  locations  of  the  least  signifi- 
cant inputs. 
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